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INTRODUCTION 

Lung  cancer  is  a  leading  cause  of  death  in  the  United  States  [1,2].  The  results  from  several  large  lung  cancer 
screening  studies  indicate  that  early  detection  and  treatment  can  reduce  mortality  rate  in  most  types  of  lung  cancer 
cases  [3-6].  Currently,  low-dose  CT  scanner  is  a  primary  tool  used  for  lung  cancer  screening.  For  each  screening 
case,  a  set  of  image  slices  covering  entire  lung  area  is  generated  and  viewed  on  workstation  monitors.  Despite  of  3D 
data  obtained  from  CT  scanner,  the  conventional  reading  method  for  lung  CT  image  interpretation  is  to  read  images 
slice-by-slice.  This  reading  method  requires  radiologists  to  mentally  reconstruct  images  in  3D  space  from  a  set  of  2D 
images  to  differentiate  normal  tubular  structures  from  nodules.  Furthermore,  with  improved  technology  for  CT 
scanner,  higher  resolution  images  produce  more  images  per  scan,  which  eventually  will  exceed  radiologists'  ability 
to  read  cases  in  slice-by-slice  mode.  The  need  of  3D  data  presentation  of  CT  images  has  become  crucial  for  ever- 
increasing  numbers  generated  from  CT  scanner  and  for  improvement  of  radiologists'  performance  on  image  data 
interpretation.  We  have  proposed  to  develop  a  stereo  display  workstation  for  reading  lung  CT  images.  Stereopsis  is 
the  mechanism  used  in  human  vision  system  to  perceive  objects  in  our  three  dimensional  space.  The  3D  display 
using  stereoscopic  projection  should  produce  a  natural  and  efficient  solution  for  3D  data  presentation.  In  this 
proposal,  we  hypothesized  that  the  efficacy  of  lung  cancer  screening,  by  CT  scanner,  can  be  increased  by  use  of  a 
suitable  designed  stereoscopic  display.  Specifically,  we  expect  that  both  efficiency,  and  accuracy  for  the  detection  of 
lung  nodules,  will  be  increased  significantly  over  what  can  be  achieved  when  reading  cases  in  currently  used  display 
modes.  To  achieve  the  goals  in  this  proposal,  we  have  specified  our  aims  in: 

1)  Develop  and  integrate  the  hardware  and  software  required  to  implement  a  stereoscopic  display  tailored  to 
chest  CT  images. 

2)  Use  a  subset  of  lung  cancer  cases,  verified  either  by  pathology  or  by  followup,  to  evaluate  the  display 
system. 

3)  Perform  a  retrospective  study  to  measure  relative  accuracy  and  reading  efficiency,  for  detection  and 
classification  of  lung  nodules,  between  three  display  modes  including  stereoscopic  3D  mode  from  this 
project,  and  other  two  commonly  used  modes,  slice-by-slice  and  maximum-intensity-projection  thick  slice. 


BODY 

Integrate  Hardware  (task  1) 

To  implement  the  stereo  display  workstation,  certain  requirements  of  hardware  components  and  hardware 
constructions  need  to  be  fulfilled  before  integration  of  the  hardware  and  software  for  viewing  stereographs. 

The  integrated  hardware  for  stereo  display  consisted  of  a  PC  computer  equipped  with  stereographic  card,  a 
programmable  keypad,  a  monitor,  a  signal  synchronizer,  and  a  shutter  glasses. 

Computer  —  A  2.8  GHz  AMD  Athlon  64  personal  computer  was  configured  for  stereo  image  processing  and 
stereographic  display.  The  computer  has  3  hard  disk  units  connected  via  RAID  technology  to  create  maximal  disk 
capacity  of  400  GH  that  can  support  massive  data  computing  and  real-time  display  for  stereo  chest  CT  workstation. 
The  NVIDIA  Quadro  FX1100  graphic  card  installed  in  the  computer  is  stereo  capable  that  provides  display  buffers 
and  OpenGL/DirectX  support  necessary  for  stereographic  presentation.  The  computer's  performance  and 
stereographic  capability  was  tested  for  the  optimal  level  to  the  tasks  in  this  project. 

Keyboard  —  Several  functionalities  for  controlling  stereo  display  window  were  encoded  into  a  programmable 
keypad.  The  key-controlled  features  include,  but  not  limited  to,  adjusting  window/level  settings,  scrolling  sequential 
images  in  a  case,  changing  number  of  image  slices  that  are  used  for  composing  stereo  images,  and  toggling  markers 
used  for  detected  nodule.  In  lieu  of  regular  keyboard,  the  keypad  we  reconstructed  is  tempered  to  the  needs  of  lung 
nodule  detection  and  classification  in  the  stereo  display.  By  simply  pushing  on  the  functional  keys,  radiologists  can 
manage  the  display  to  the  level  of  optimal  viewing  and  task  specific. 

Stereo  output  —  Three  parts,  monitor,  signal  emitter  and  shutter  glasses,  in  the  integrated  hardware  are  mainly 
responsible  for  stereo  image  output.  In  order  to  perceive  stereoscopic  view  of  the  images,  left  and  right  images  need 
to  be  viewed  separately  by  each  of  corresponding  eyes  at  frame  rate  of  at  least  60  HZ  for  each  eye.  The  graphic  card 
was  configured  to  alternate  displayed  left  and  right  images  and  output  the  signal  through  the  emitter  to  synchronize 
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the  shutter  glasses  to  the  refresh  cycle  of  the  monitor.  The  monitor  with  refresh  rate  of  144  HZ  is  used  to  give 
flicker-free  stereo  images  viewed  through  shutter  glasses. 

Develop  3D  Geometric  Projection  Algorithms  and  Transparency-Contrast  Models  (tasks  2,  3) 

Display  a  3D  chest  CT  dataset  in  stereo  requires  that  two  2D  projections,  corresponding  to  left  and  right  eye  views 
(stereo  pair),  be  performed.  Projection  methods  consist  of  geometric  projection  model  and  illumination  model.  The 
interaction  between  geometric  projection  model,  illumination  model  and  optical  characteristics  at  each  voxel  can  be 
integrated  along  rays  to  calculate  the  values  of  pixels  on  2D  projection.  For  CT  images,  it  is  customary,  for  display, 
to  assign  brightness  values  having  a  monotonic,  but  nonlinear  relationship  to  X-ray  attenuation  coefficients,  as 
measured  in  Hounsfield  units.  In  keeping  with  this,  we  assumed  that  each  CT  voxel  has  a  neutral  color  (some  shade 
of  gray)  and  a  brightness  value  that  is  some  affine  mapping  of  its  Hounsfield  value,  with  pixels  corresponding  to  low 
X-ray  attenuation  appearing  as  darker  intensities  and  pixels  corresponding  to  higher  X-ray  attenuation  appearing 
brighter. 

In  stereopsis,  contrast  is  important  for  depth  perception  [7-8],  and  subjective  evaluations  from  our  studies  indicate 
that  optimization  of  local  contrast  is  necessity  for  detecting  fine  structures  in  stereo  displays  of  chest  CT  data. 
Monocular  occlusions  from  the  perspective  transformation  and  from  averaging  effects  of  transparency  models 
incorporated  in  traditional  volumetric  reconstruction  algorithms  possibly  reduces  contrast  in  the  reconstructed  stereo 
pairs  with  increased  image  depth  from  stacks  of  CT  slices.  Our  goal,  therefore,  was  to  maximize  the  detectability  of 
nodules,  but  at  the  same  time,  maintain  sufficient  geometric  fidelity  to  allow  detected  nodules  to  be  accurately 
characterized.  The  issues  of  clearly  displaying  other  structures  of  interest  such  as  vessels,  bronchi  and  bone  and 
incorporation  of  monoscopic  depth  cures  were  also  addressed  and  implemented  in  the  study. 

a.  Geometric  Models 

The  two  geometric  projection  models  that  are  relevant  in  the  context  of  this  application  are  the  orthographic  and  the 
perspective  transformations.  Orthographic  projection,  which  is  used  by  traditional  Maximum  Intensity  Projection 
(MIP),  is  not  compatible  with  realistic  stereographic  projection.  Perspective  transformations  have  been  used  to  a 
limited  extent  in  the  medical  environment  [9],  but  their  added  complexity  is  not  always  justified  for  monoscopic 
viewing.  Geometric  perspective  is  one  of  the  more  important  monoscopic  depth  cues  when  familiar  scenes  are  being 
viewed,  though  the  extent,  to  which  it  will  be  of  value  for  viewing  the  interior  of  lungs,  is  not  known. 

To  test  the  prospective  use  of  perspective  transformation  for  stereo  view  of  lung  images,  we  applied  perspective 
transformation  in  stereo  image  compositing  algorithm  to  a  set  of  chest  CT  images.  We  assumed  that  the  topmost 
slice  being  displayed  is  the  same  distance  as  the  screen  of  the  physical  display,  the  viewing  distance  between  a 
viewer  and  the  screen  is  45-cm,  and  the  interocular  distance  is  6.5-cm.  We  have  maintained  these  conditions 
throughout  this  work,  but  in  our  studies  we  found  that  stereoscopic  convergence  is  not  appreciably  affected  by  the 
exact  seating  position  —  as  long  as  the  eyes  are  maintained  level  relative  to  the  monitor. 

It  is  generally  aware  that  volumetric  lung  images  acquired  by  current  CT  systems  have  different  resolutions  in  the 
axial  direction  than  in  planes  perpendicular  to  that  direction.  The  axial  resolution  is  more  a  function  of  the  user's 
specification  to  the  reconstruction  algorithm,  than  to  the  acquisition  protocol.  The  anisotropic  nature  of  the  dataset 
(e.g.,  voxels  are  typically  0.7-mm  x  0.7-mm  x  2.5-mm),  and  our  choice  of  a  non-orthogonal  projection,  necessitates 
resampling  the  data  to  obtain  the  actual  values  used  in  ray  casting.  The  most  common  method  used  for  interpolation 
in  computer  graphics  is  trilinear  interpolation  because  it  is  easy  to  implement,  computational  efficient  and 
comparable  to  other  more  sophisticated  interpolation  methods  in  terms  of  visual  effect.  We  used  trilinear 
interpolation  to  resample  the  data  and  insert  effectively  the  virtual  slices  between  each  pair  of  real  slices  to  achieve 
final  pixel  dimension  close  to  isotropic. 

The  results  from  MIP  volumetric  rendering  with  perspective  transformation  produced  stereo  images  that  had  true  3D 
depth  and  spatial  differentiation  of  interior  lung  structures.  Comparing  visually  to  perspective  transformation,  MIP 
rendering  with  orthogonal  transformation  generated  only  monoscopic  3D  view  and  no  information  of  depth  or  of  the 
geometrical  relationships  between  structures.  However,  further  quantitative  comparison  of  these  two  transformations 
and  their  utility  in  medical  image  diagnosis,  chest  CT  in  particular  in  our  case,  will  be  performed  in  the  upcoming 
observer  performance  studies  (year  two). 
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b.  Implement  monoscopic  depth  cues 

Disparity  between  stereoscopic  views  is  the  primary  depth  cue  being  studied  in  this  work.  However,  there  exist 
monoscopic  depth  cues  that  greatly  assist  viewers  in  achieving  stereopsis,  and  it  is  desirable  to  include  these  cues,  to 
the  extent  they  are  not  in  conflict  with  the  goal  of  optimizing  radiographic  interpretation.  Two  monoscopic  cues  were 
incorporated  in  the  stereo  compositing  methods,  geometric  depth  cues  and  brightness  cue. 

Geometric  cues  —  The  main  geometric  depth  cue  we  have  included  was  geometric  perspective  which,  as  was 
discussed  above,  was  the  transformation  adopted  exclusively  for  rendering  stereoscopic  images  in  this  project.  A 
second  geometric  cue  we  have  included  was  the  occlusion  of  voxels  by  intervening  structures  to  determine  the 
amount  of  illumination  projected  on  display  plane  for  an  object. 


Brightness  cue  —  We  have  modeled  brightness  variation  with  respect  to  depth  change.  To  model  a  systematic 
reduction  in  brightness  values  with  increasing  depth  or  distance,  we  assumed  that  each  slice  has  a  fixed  optical 
density  that  reduces  the  brightness  of  slices  lying  behind  it.  In  the  actual  implementation,  each  slice  was  assumed  to 
have  the  same  fixed  optical  density  and  weighing  factors,  derived  from  a  geometric  sequence,  was  applied  so  as  to 
achieve  a  ratio  of  weights,  K,  between  the  back  slice  and  the  front  slice.  For  N  slices,  and  assuming  the  total  of  all 
weights  equals  1,  the  weights  W\,  i=0,  ...  ,  N-l,  was  calculated  as  follows: 


1 

1-^iV-l  — 

W  =  Ar  kn~ 1 

i  N 

\-KN^ 


In  the  case  of  our  distance-weighted  MIP  projections,  the  front  slice  weight  was  1,  and  the  sum  was  irrelevant.  In  our 
preliminary  studies  we  found  that  K= 0.5  provides  a  reasonable  indication  of  depth  without  excessively  reducing  the 
brightness  of  the  deeper  slices. 

c.  Illumination  Models 

Volume  rendering,  which  was  adopted  for  our  stereo  image  compositing,  attempts  to  identify  and  classify  all  voxels 
of  an  object,  and  to  assign  optical  properties  (e.g.,  transparency,  color  and  brightness)  to  each  voxel  and  is 
particularly  useful  in  applications,  such  as  chest  CT,  where  the  structures  of  interest  are  rather  sparsely  distributed. 

The  challenging  task  in  volume  rendering  is  to  develop  an  optimal  brightness  /  contrast  /  transparency  model  for 
assigning  optical  characteristics  to  each  rendered  voxel.  Because  the  intrinsic  property  being  imaged  with  CT  is 
actually  X-ray  attenuation  coefficient,  the  assignment  of  optical  properties  to  CT  voxels  is  entirely  artificial  and 
should  be  done  in  a  manner  that  best  facilitates  the  interpretation  of  the  image  data.  Two  illumination  models  that  are 
suitable  for  chest  CT  data  have  been  examined  and  compared. 

1 .  Averaging  model  —  this  compositing  method  that  is  commonly  applied  in  volume  rendering  averages  voxel  values 
along  rays  and  effectively  reduces  the  contrast  of  smaller  objects,  which  may  appear  in  only  a  few  slices,  as  the 
displayed  volume  is  increased  by  adding  slices.  Nevertheless,  volume  rendering  with  averaging  does  preserve  spatial 
information  such  as  textures  and  local  geometry. 

For  chest  CT  images,  instead  of  simply  averaging  voxel  values  along  rays,  we  adopted  a  light  emission  / 
transmission  /  occlusion  model  that  assumed  that  each  voxel  emits  light  in  proportion  to  its  brightness  when  CT 
images  are  displayed  at  a  normal  window  and  level  for  nodule  detection  and  that  uses  distance  information  (distance¬ 
weighting  factors)  to  determine  the  amount  of  this  emitted  light  that  reaches  the  projection  plane  (appendix  A). 
Specifically,  it  was  assumed  that  each  slice  has  a  fixed  optical  density  that  decreases  the  brightness  of  slices  lying 
behind  it.  The  total  of  all  distance  weights  was  equal  to  one.  The  ratio  of  weights  between  the  last  (the  slice  with  the 
largest  distance  from  screen)  and  first  slice  (the  slice  at  screen  level)  controls  the  level  of  transparency  for  a  given 
volume. 
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We  studied  a  range  of  these  ratios  for  lung  CT  images  and  empirically  set  the  ratio  to  0.5  to  achieve  a  balance 
between  the  use  of  brightness  weighting  as  a  depth  cue  and  visibility  of  the  back  slice.  The  final  value  for  a  voxel  is 
the  sum  of  distance-weighted  pixel  values  in  a  perspective  transformation  ray. 

2.  MIP  models  —  by  using  the  maximum  pixel  intensity  along  each  ray  of  the  projection,  MIP  is  designed  to 
maximize  contrast  in  situations  where  sparsely  distributed  objects  are  being  viewed  against  a  dark  background, 
which  is  the  situation  that  occurs  in  projecting  thick  volumes  of  the  lung.  Its  main  deficiency  is  that  it  does  not 
preserve  local  spatial  structure  such  as  texture  and  local  geometry.  This  is  because  a  bright  pixel  appearing  in  a  view 
for  one  eye  will  not  necessarily  have  a  corresponding  bright  pixel  that  appears  in  the  view  for  the  opposite  eye. 

In  our  stereographic  compositing  with  MIP  principle,  we  have  experimented  two  different  ways  of  applying  MIP 
compositing.  First,  we  used  a  perspective  projection  in  which  the  maximum  value  along  each  ray  was  used  as  the 
projected  value.  As  we  mentioned  above,  this  approach  generally  is  not  possible  for  an  observer's  vision  system  to 
unambiguously  match  corresponding  points  between  the  two  views  because  the  projected  voxels  may  be  different 
between  the  stereo  views.  In  practice,  we  found  that  the  ambiguity  in  matching  corresponding  points  between  views 
primarily  affects  fine  detail  and  does  not  interfere  with  the  detection  of  objects  composed  of  large  clusters  of  bright 
voxels,  although  the  exact  shapes  of  these  objects  cannot  be  determined  unambiguously. 

In  an  attempt  to  incorporate  a  geometric  cue  common  in  traditional  stereo  projection  methods,  but  also  preserve  the 
contrast  advantage  of  MIP,  we  in  turn  used  a  perspective  projection  in  which  the  maximum  along  each  ray  has  been 
weighted,  based  on  distance.  The  maximum  (nearest)  and  minimum  (farthest)  weights  were  determined  empirically, 
and  weights  for  slices  between  the  first  and  last  were  calculated  by  suing  a  geometric  sequence,  assuming  a  fixed 
optical  density  for  each  slice. 

d.  Compare  compositing  methods  for  lung  nodule  detection  and  characterization  on  stereo  CT  images 
We  compared  averaging,  MIP  and  distance-weighted  MIP  applied  to  various  lung  nodule  types,  sizes,  and  locations. 
MIP  and  distance-weighted  MIP  produced  higher  local  contrast  than  compositing  by  averaging.  Unlike  averaging 
methods,  which  sacrifice  contrast  to  take  account  of  each  voxel  in  a  volume,  the  conventional  MIP  method  is  able  to 
retain  contrast  in  cases  in  which  the  object  being  viewed  includes  voxels  that  are  brighter  than  the  superimposed 
tissue.  Applied  to  the  task  of  lung  nodule  detection,  despite  a  lack  of  geometric  fidelity,  conventional  MIP  images 
generally  produce  high  local  contrast  that  separates  a  nodule  from  its  background  and  therefore  enhances  detection 
performance. 

However,  improvements  in  nodule  visibility  with  the  MIP  method  do  not  apply  in  certain  cases  in  lung  CT  images. 
For  a  nodule  to  be  detected  with  conventional  MIP,  it  must  contain  some  voxels  that  are  brighter  than  its 
background.  The  case  of  a  nodule  overlying  a  rib  is  a  particular  concern,  although  it  occurs  relatively  infrequently  in 
projections  of  axial  slabs  unless  the  nodule  is  very  close  to  a  rib  or  the  slab  being  viewed  is  relatively  thick.  Most 
voxels  in  such  a  nodule  will  not  be  as  bright  as  voxels  in  the  rib,  and  the  nodule  may  be  almost  invisible. 

One  benefit  of  the  distance-weighted  MIP  projection  is  that  it  can  decrease  the  brightness  of  the  background.  As  the 
position  of  a  slab  is  changed,  relative  brightness  weighting  factors  between  voxels  at  different  axial  positions  will 
change,  and,  in  many  instances,  there  will  be  an  axial  position  at  which  a  nodule  will  appear  brighter  than  a  rib, 
which  can  increase  the  likelihood  that  an  obscured  nodule  becomes  visible.  Our  results  indicate  that  distance- 
weighted  MIP  partially  recovered  geometric  information  lost  in  conventional  MIP  by  incorporating  a  distance  cue 
into  the  compositing,  and  at  the  meantime,  distance-weighted  MIP  had  image  contrast  nearly  equivalent  to  that 
produced  by  the  conventional  MIP  method. 

In  addition  to  detection,  nodule  characteristics  are  essential  and  critical  for  clinically  differentiating  benign  for 
malignant.  Although  conventional  MIP  is  superior  for  detection,  it  was  outperformed  by  the  averaging  method  in 
terms  of  characterization.  When  comparing  the  three  compositing  methods  visually  for  nodule  characterization,  the 
spiculated  nodule  border  is  clearly  visible  in  stereo  pairs  composited  with  the  averaging  method,  whereas  this 
characteristic  is  not  preserved  in  some  of  the  stereo  pairs  composited  with  the  conventional  MIP  method,  especially 
those  composited  from  thicker  slabs.  Distance-weighted  MIP  partially  overcomes  the  problem  of  conventional  MIP, 
and  speculated  borders  are  still  visible  in  the  thicker  slabs.  For  the  smooth  borer  of  a  nodule,  we  observed  a  similar 
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phenomenon.  The  geometric  relationship  is  well  presented  with  gradient  changes  in  intensities  along  the  smooth 
border  of  the  nodule  in  the  stereo  pair  composted  with  the  averaging  method.  The  same  nodule  is  shown  with  lack  of 
geometric  fidelity  in  the  stereo  pair  composited  with  conventional  MIP.  The  lack  of  fidelity  in  module  shape  and 
geometric  representation  in  conventional  MIP  images  are  attributed  to  the  nature  of  MIP  compositing,  in  which  the 
two  views  may  be  based  on  projections  of  different  voxels.  Conversely,  the  averaging  method  is  shown  in  this  study 
to  faithfully  retain  the  characteristics  of  the  nodules,  including  structural,  spatial,  and  geometric  information. 

e.  Test  combinations  of  MIP  and  averasins 

Each  compositing  method  as  described  above  has  a  particular  balance  of  advantages  and  disadvantages,  and  may  be 
optimal  in  certain  situations  but  inappropriate  for  others.  Based  on  our  preliminary  results,  we  believe  that  two 
rendering  modes  will  need  to  be  available  to  viewers  —  one  optimized  for  nodule  detection  and  one  optimized  for 
nodule  characterization.  Thus,  in  practice,  it  may  actually  be  advantageous  to  view  a  volume  with  a  range  of 
compositing  methods. 

Because  MIP  appears  to  be  best  for  nodule  detection,  and  some  form  of  voxel  averaging  is  best  for  charactering 
detected  nodule,  we  have  implemented  and  used  two  separate  rendering  methods,  i.e.  distance-weighted  MIP  and 
averaging,  in  a  single  stereoscopic  display  mode.  The  intention  is  that  after  a  nodule  has  been  detected  in  the 
distance-weighted  MIP,  the  thickness  of  the  displayed  volume  can  be  adjusted  to  include  only  slices  that  contain  the 
nodule,  and  this  volume  can  be  displayed  by  using  an  averaging  compositing  method  so  the  nodule  can  be 
characterized  more  accurately. 

Write  Display  Software  (task  4) 

The  display  software  provided  for  the  stereo  viewing  has  been  developed  in  the  current  progress  period.  The 
functionalities  built  into  the  software  enabled  readers  to  have  real-time  interaction  with  slice  navigation  through 
entire  lung  sections,  to  change  viewing  thickness  of  lung  sections,  and  to  adjust  brightness  /  contrast  of  displayed 
images  (Appendix  B). 

a.  Write  user  interface 

The  display  software  was  written  as  a  Windows  application  implemented  specifically  with  Win32  including  MFC 
(Microsoft  Foundation  Classes),  and  SGI  (Silicon  Graphics  Inc.)  OpenGL  (Open  Graphics  Library)  language.  The 
Windows  API  provided  Windows  framework  and  functional  utilities  of  the  user  interface.  The  OpenGL  functions 
enabled  stereographic  display  by  manipulating  display  functions  on  graphic  processing  unit.  A  typical  user  interface 
is  shown  in  figure  1 . 


Figure  1.  Window-based  user  interface. 
b.  Write  routines  for  prestaging 

projections 
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In  order  to  have  users  to  control  the  thickness  (i.e.,  number  of  slices  in  a  slab)  and  the  axial  position  of  stacks  of  2D 
CT  slices,  all  projections  were  precalculated  and  stored  on  the  display's  hard  disk.  For  each  case,  the  files  were 
organized  into  a  two-dimensional  linked  list  that  allows  projections  to  be  accessed  by  thickness  and  axial  position  in 
real-time.  Software  was  written  to  render  stereo  image  pairs  from  the  CT  data  by  the  stereo  projection  algorithms 
described  in  tasks  2,  3. 

For  the  comparisons,  the  software  also  included  rendering  traditional  monoscopic  MIP  images.  To  be  consistent  with 
commercially  available  CT  displays,  monoscopic  MIP  images  were  generated  using  a  standard  orthogonal 
projection.  The  slice-by-slice  display  mode  required  only  minimal  additional  software  because,  by  design,  this  mode 
is  actually  a  limiting  case  (i.e.,  volume  thickness  =  1  slice)  of  both  the  MIP  and  stereo  modes. 

All  computer  code  was  written  in  C++  language  and  tested  thoroughly  prior  to  its  use  in  this  study. 

c.  Write  routines  for  window/level  control 

Window  /  level  (contrast  /  brightness)  mechanism  was  built  into  the  display  workstation  with  a  default  setting  of 
2000  /  -500-HU  for  lung  CT  images.  Real-time  adjustment  of  window  /  level  was  enabled  through  encoded  keys  on 
the  keypad. 

d.  Write  routines  for  control  of  slice  thickness  and  axial  position 

To  achieve  real-time  performance  in  changing  axial  position  in  the  CT  volume  and  in  changing  the  thickness  of  the 
displayed  volume,  stereo  pairs  corresponding  to  all  admissible  combinations  were  precalculated,  organized  into  a  2D 
linked  list,  and  stored  on  the  hard  drive.  This  required  approximately  1-GB  of  disk  storage  per  case,  but  was  able  to 
achieve  nearly  instantaneous  response  to  changes  in  axial  position  or  slice  thickness. 

Write  Software  Needed  To  Perforin  Study  (task  6) 

In  the  main  study  as  well  as  the  pilot  study,  readers  are  required  to  identify  possible  nodules  and  then  characterize 
the  detected  nodules  in  three  display  modes.  A  software  was  written  to  implement  functions  required  for  the  study, 
which  include  management  of  study  cases  and  display  modes,  case  randomization,  electronic  scoring  of  detected 
nodules,  scaled  onscreen  ruler  for  nodule  measurement,  markers  for  detected  nodules,  and  data  archive.  All 
implementations  were  written  in  C++  language  and  have  been  tested  rigorously  to  meet  the  specifications  of  the 
study. 

a.  Write  3D  cursor  routine  for  making  nodule  locations 

We  were  able  to  make  a  stereo  cursor  corresponding  to  the  perspective  view  of  displayed  volume.  In  specific,  the 
cursor  can  be  specifically  placed  at  a  particular  location  in  the  3D  volume  space.  The  size  of  the  cursor  is  properly 
transformed  to  reflect  the  size  perceived  at  a  distance  based  on  the  spatial  location  of  the  cursor  in  the  volume.  With 
the  stereo  cursor,  a  reader  can  mark  a  nodule  location  (with  3  axes  of  x,  y,  and  z)  in  the  volume.  At  the  present,  we 
have  placed  the  cursor  in  the  middle  of  the  viewing  slab.  To  make  a  navigable  stereo  cursor,  we  just  need  to  encode 
mouse  wheel  with  the  stereo  cursor  routines,  since  all  the  mechanisms  have  been  laid  out. 

Besides  serving  as  a  location  indicator,  the  stereo  cursor  was  also  made  as  an  onscreen  measure  for  nodule  size. 
Horizontal  or  vertical  line  of  the  cursor  represents  unit  of  length  that  can  be  used  for  size  approximation  when  the 
cursor  is  placed  over  a  structure. 

b.  Implement  case  randomization 

In  observer  performance  study,  case  randomization  is  a  procedure  to  assure  that  a  reader's  response  is  not  biased  by 
the  order  and  the  time  of  the  cases  presented.  Our  randomization  routines  were  designed  to  generate  random  case 
list,  remove  the  cases  that  have  been  examined  and  then  present  the  cases  for  each  reader  at  each  reading  session. 
Because  of  the  multiple  modes  we  need  to  test  in  this  study,  the  randomization  routines  have  also  implemented 
function  that  checks  status  of  a  case  across  the  modes. 


c.  Record  findings 
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Electronic  scoring  form  was  implemented  in  the  study  software.  When  a  reader  clicks  on  a  possible  nodule,  a  scoring 
form  with  study  questions  will  show  on  the  screen.  The  questionnaire  can  be  answered  in  the  form  of  either  check 
boxes  or  radio  buttons.  The  answers  as  well  as  nodule's  location  will  be  saved  after  reader  finishes  scoring  or  will  be 
neglected  if  reader  chooses  to  delete  the  answers.  A  detected  nodule  is  toggled  with  a  maker  and  on/off  states  of  the 
marker  can  be  controlled  at  reader's  will. 

d.  Write  routines  for  monitoring  readers'  reading  patterns 

It  is  our  intention  to  incorporate  routines  into  the  study  software  for  keeping  track  of  moves  made  by  readers  during 
case  study.  This  data  will  be  a  part  of  evaluation  criteria  for  the  display  study  as  well  as  be  valuable  reference  for 
understanding  psychophysical  attributes  in  different  display  designs.  During  a  reading  session,  the  computer  will 
periodically  (at  the  rate  of  every  5th  millisecond)  record  reading  status,  including  anatomic  location  of  displayed 
slice,  displayed  slice  volume  in  the  case  of  stereo  or  M1P  mode,  indication  of  status  between  observing  case  and 
scoring  nodule,  and  indication  of  compositing  methods  between  averaging  and  MIP  in  the  case  of  stereo  mode. 

Case  Selection  (task  7) 

The  images  selected  for  developing  and  evaluating  the  display  were  from  subjects  who  have  previously  been 
scanned  as  part  of  the  Pittsburgh  Lung  Cancer  Screening  Study,  an  ongoing  lung  cancer  screening  trial  in  our 
facility,  of  subjects  that  are  considered  be  at  high  risk  due  to  age  and  smoking  history.  The  cases  were  acquired  from 
a  helical  CT  scanner  (LightSpeed  Plus,  GE  Medical  Systems,  Milwaukee,  WI)  with  slices  reconstructed  at  a 
thickness  of  2.5-mm,  which  resulted  in  the  production  of,  on  average,  100  slices  per  case.  The  images  were  post- 
processed  with  a  kernel  convolution  algorithm  using  GE  standard  software,  to  adjust  image  sharpness  to  be  suitable 
for  viewing  lung  tissue. 

The  cases  we  selected  were  from  a  population  where  lung  nodules,  of  which  most  are  noncancerous,  are  prevalent 
due  to  subjects'  age  and  smoking  history.  The  average  number  of  nodules  per  case  is  about  five.  The  actual 
distribution  of  nodules  in  our  cases  produces  a  somewhat  lower  sensitivity,  for  case-based  analysis,  than  we  were 
expecting.  The  high  probability  of  finding  positive  cases  would  provide  low  sensitivity  and  less  statistical  power  for 
comparing  radiologists'  performance  of  nodule  detection  across  the  different  display  modes,  if  we  conduct  case- 
based  comparison  study.  To  enhance  the  analytical  power  of  the  data,  we  will  adopt  an  observation-based  strategy  as 
the  primary  method  of  analysis.  That  is,  the  performance  will  be  analyzed  based  on  each  finding  in  a  case  and 
analyzed  by  FROG  methods.  The  change  in  emphasis  from  a  case-based  study  to  an  observation-based  study  means 
a  significant  increase  in  statistical  power. 

Meanwhile,  we  have  included  more  readers  in  both  the  pilot  study  and  the  main  study.  Particularly,  we  have  more 
readers,  especially  recruiting  fellows,  so  as  to  reflect  a  spectrum  of  various  levels  of  clinical  experience  of  readers. 
This  is  an  important  issue  for  us,  as  we  need  to  know  if  any  performance  change  has  any  association  with  subjects' 
experience  and,  to  the  less  extent,  subjects'  physical  conditions  (such  as  age,  visual  conditions),  typically  when 
stereoscopic  vision  and  eyewear  are  involved  in  the  study.  The  number  of  readers  is  increased  from  3  to  6  in  the  pilot 
study,  and  from  6  to  8  in  the  main  study.  This  increase  of  the  number  of  readers  gives  further  justification  of 
reducing  the  number  of  cases  we  initially  proposed. 

As  stated  above,  redefining  the  definition  of  study  instance  and  adjusting  reader  structure  allow  achieving  at  least  the 
same  analysis  power  but  fewer  cases.  As  the  result  of  that,  the  final  number  of  cases  we  collected  for  this  project  is 
reduced  to  290,  which  contain  total  of  1630  consensus  nodules,  in  which  18  are  highly  suspicious  and  10  are 
pathologically  proved  cancer. 

a.  Nodule  verification 

Like  other  cancers,  biopsy  for  lung  cancer  is  reserved  for  highly  suspicious  ones,  but  unlike  other  cancer  diagnosis, 
lung  nodules  are  highly  prevalent  in  the  screening  population  and  most  of  the  nodules  are  often  indolent.  Thereby, 
most  detected  lung  nodules  are  left  for  followup  without  immediate  biopsy.  In  our  collected  cases,  only  10  out  of 
1630  nodules  were  pathologically  proved  to  be  malignant  and  the  rest  of  the  nodules  are  diagnosed  based  on  clinical 
impressions.  To  obtain  truth  of  nodule  characteristics,  we  used  disease  free  follow-up  for  negative  nodule 
verification  and  meantime  we  adopted  the  consensus  method,  which  is  widely  used  and  accepted  in  the  scientific 
groups  for  lung  nodule  verification  [10]. 
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To  obtain  consensus  results,  three  experienced  thoracic  radiologists  have  reviewed  all  the  nodules.  The  review 
process  included  nodule  identification,  verification,  and  characterization.  Such  review  process  was  repeated  at  least 
once  to  ensure  agreement  on  nodule  identification  and  characterization.  Table  1  lists  all  fields  of  a  nodule 
description. 

Table  1. 


Name 

Specifications 

Nodule  number 

Number  of  Nodules 

Nodule  size 

Measured  in  x,  y  axes  (mm) 

Nodule  primary  character 

solid;  non-solid  (ground  glass);  mixed  solid  and  non-solid 

Nodule  borders 

smooth;  spiculated;  lobulated 

Calcifications 

absent;  present 

Location  (side) 

right  lung;  left  lung 

Location  (lobe) 

Right  lung:  RUL;  RML;  RLL 

Left  lung:  LUL;  LLL 

CT  image  number 

CT  series 

Risk  level 

Probability  of  malignancy 

b.  Cases  anonymization  by  honest  broker 

To  comply  with  HIPPA  regulations,  the  case  collection  process  was  solely  conducted  by  an  honest  broker,  who  has 
no  any  association  with  this  project.  After  the  cases  had  been  selected,  all  personal  information  were  removed  from 
the  cases  and  the  cases  were  renamed  with  study  numbers. 

Reader  Training  (task  8) 

Participating  radiologists  receive  an  "Instructions  for  Observers"  form  for  review  (Appendix  C),  and  the  definition  of 
abnormalities  are  discussed  with  each  reader  prior  to  both  the  pilot  and  main  studies.  Readers  were  also  trained  on 
the  use  of  our  scoring  mechanism  during  training  session.  To  verify  that  observer  performance  is  not  affected  by 
relative  unfamiliarity  or  familiarity  with  our  system,  we  will  compare  performance  of  each  radiologist  at  the 
beginning,  middle,  an  end  of  the  study  once  we  complete  the  data  collection  from  the  pilot  study. 

Perform  Pilot  Study  (task  9) 

The  pilot  study  was  organized  as  a  retrospective  study  of  108  nodules  in  30  cases.  Six  radiologists,  4  experienced 
radiologists  and  2  fellows,  have  participated  the  study.  The  reading  data  has  been  collected  and  is  being  analyzed  for 
the  performance.  This  will  provide  an  opportunity  to  further  refine  other  aspects  of  the  definitive  study. 

KEY  RESEARCH  ACCOMPLISHMENTS 

■  Developed  the  mechanisms  for  a  real-time  stereographic  display  for  volumetric  datasets  of  lung  CT  images 

■  Developed  study  software  for  investigating  observer  performance  on  stereo  display  and  other  commonly  used 
displays  for  lung  CT  images 

■  Developed  3D  geometric  projection  algorithms  and  transparency-contrast  models  for  compositing  of  stereo 
images  from  volumetric  lung  CT  data 

■  Collected  290  cases,  which  include  1630  nodules  with  28  that  are  highly  likely  as  malignancy. 

■  Trained  readers  for  performing  observer  performance  study- 

■  Performed  the  pilot  study  and  collected  the  data  for  further  analysis 


REPORTABLE  OUTCOMES 

Peer  reviewed  paper 
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Stereo  CT  image  compositing  methods  for  lung  nodule  detection  and  characterization.  Academic  Radiology, 

2005  (in  print). 


Abstract: 

Real-time  stereographic  display  of  volumetric  datasets  in  radiology.  SPIE,  Electronic  Imaging,  2006  (accepted). 
Grant  application: 

Optimizing  MDCT  display  for  detection  and  diagnosis  of  pulmonary  embolism.  Summated  to  NIH,  June  1, 
2005. 

Real-time  stereo  projection  and  display  for  3D  radiographic  data.  Submitted  to  NIH,  June  17,  2005. 

Real-time  interactive  display  for  virtual  endoscopy.  Submitted  to  NIH,  October  19,  2005. 


CONCLUSIONS 

Our  primary  objective  is  to  determine  whether  a  stereoscopic  display  concept  has  potential  for  improving  the 
efficiency  and  accuracy  of  chest  CT  for  lung  cancer  screening.  In  this  progress  period,  our  main  tasks  were  to 
develop  algorithms,  software  infrastructures  and  to  prepare  hardware  and  study  cases,  which  are  all  necessary  for  the 
study. 

Mainly,  we  have  investigated  and  developed  algorithms  to  generate  stereo  image  pairs  from  volumetric  chest  CT 
images.  More  specifically,  we  have  used  combined  models  of  geometric  transformation  and  transparency-contrast 
for  develop;  ping  compositing  methods  that  were  specially  tailored  to  stereo  image  pairs  from  hung  CT  images.  For 
preparing  the  study,  a  software  was  developed  to  include  the  functionalities  needed  for  the  observer  performance 
study.  In  addition,  we  have  built  a  display  workstation  that  can  stereoscopically  display  chest  CT  images  in  real¬ 
time,  with  mechanisms  for  various  user  interactive  actions. 

The  important  issue  we  have  addressed  in  this  progress  period  is  the  optimal  rendering  methods  for  nodule  detection 
in  lung  cancer  screening  with  stereographic  display.  Choosing  a  compositing  method  that  is  optimal  for  a  particular 
task,  lung  nodule  in  our  case,  is  the  key  decision  in  optimizing  stereographic  displays.  Our  results  indicate  that 
distance-weighted  MIP  may  be  a  better  choice  for  lung  nodule  detection  in  stereo  lung  CT  images  for  its  high  local 
contrast  and  partial  preservation  of  geometric  information,  whereas  compositing  by  means  of  distance-weighted 
averaging  is  preferable  for  nodule  characterization.  The  relative  clinical  value  of  these  compositing  methods  will  be 
evaluated  in  the  upcoming  data  analysis  and  the  main  study. 
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ABSTRACT 

Rationale  and  Objectives  Stereographic  display  has  been  proposed  as  a  possible  method  of  improving  performance 
in  reading  CT  exams  acquired  for  lung  cancer  screening.  Optimizing  such  displays  is  important,  given  the  large 
volume  of  image  data  that  must  be  evaluated  for  each  of  these  exams.  This  study  was  designed  to  explore  certain 
tradeoffs  between  rendering  methods  designed  for  the  stereo  display  of  CT  images. 

Materials  and  Methods  Stereo  CT  image  compositing  methods,  including  distance-weighted  averaging,  distance- 
weighted  Maximum  Intensity  Projection  (MIP)  and  conventional  MIP,  were  applied  to  lung  CT  images,  and 
compared  for  lung  nodule  detection  and  characterization. 

Results  The  Jonckheere  test  indicated  that  there  was  a  statistically  significant  (p<0.01)  increase  in  contrast  among 
the  three  compositing  methods.  The  Wilcoxon-Mann- Whitney  test  showed  significant  differences  in  contrast 
between  distance-weighted  averaging  and  conventional  MIP  (p<0.01)  and  between  averaging  and  distance-weighted 
MIP  (p<0.05),  but  not  between  distance-weighted  MIP  and  conventional  MIP  (p>0.05).  The  conventional  MIP 
compositing  was  found  to  provide  the  highest  image  contrast  but  produced  ambiguities  in  local  geometric  detail  and 
texture  while  the  averaging  resulted  in  the  lowest  contrast  but  preserved  geometric  detail.  Distance-weighted  MIP 
partially  recovered  geometric  information,  which  was  lost  in  the  images  composited  with  conventional  MIP. 
Conclusion  Our  results  indicate  that  distance-weighted  MIP  may  be  a  better  choice  for  nodule  detection  in  stereo 
lung  CT  images  for  its  high  local  contrast  and  partial  preservation  of  geometric  information,  while  compositing  by 
distance-weighted  averaging  is  preferable  for  nodule  characterization.  The  relative  clinical  value  of  these 
compositing  methods  needs  to  be  further  evaluated. 


INTRODUCTION 

Lung  cancer  is  a  leading  cause  of  cancer  death  in  both  men  and  women  in  the  United  States  [1],  [2],  It  has  been 
shown  that  early  detection  and  treatment  can  effectively  reduce  mortality  from  most  types  of  lung  cancers  [3],  [4], 
[5],  [6].  In  several  large  lung  cancer  screening  trials,  low-dose  helical  computer  tomography  (CT)  has  proven  to  be 
an  effective  tool  for  lung  cancer  screening  with  superior  sensitivity  (80  ~  90%)  for  early  detection,  compared  to 
other  methods  (e.g.,  chest  radiographs  with  about  23%  sensitivity  and  sputum  cytology  with  10  ~  20%  sensitivity) 
[3],  [7],  [8],  [9],  [10].  Despite  the  advantages  of  using  CT  as  a  screening  tool  for  lung  cancer,  current  methods  for 
displaying  and  interpreting  chest  CT  data  are  inefficient  and  inadequate. 

At  present,  the  two  most  common  viewing  methods  employed  by  radiologists  for  interpreting  these  studies  involve 
either  reading  individual  images  in  a  sequential  slice-by-slice  mode,  or  viewing  thicker  slabs  comprised  of  multiple 
sequential  slices  projected  onto  a  2D  display.  The  slice-by-slice  method  makes  it  necessary  for  radiologists  to 
mentally  reconstruct  3D  information  represented  in  sequential  2D  slices  in  order  to  differentiate  between  nodules 
and  linear  structures,  such  as  blood  vessels,  passing  through  the  slices.  In  addition,  the  signal-to-noise  ratio  in  single 
slices  may  be  too  low  for  subtle  lesions  to  be  reliably  detected. 

The  use  of  thicker  slabs,  obtained  by  combining  thin  slices  and  then  projecting  this  volume  onto  a  2D  display, 
partially  solves  these  problems  but  introduces  other  issues.  The  thicker  the  volume  being  projected,  the  more  likely  it 
is  that  the  superimposed  tissues  in  the  projection  will  result  in  ambiguities  in  identifying  structures  within  the 
projected  volume.  Furthermore,  if  slabs  are  projected  by  some  form  of  averaging  of  all  voxels  projected  onto  a  pixel, 
the  contrast  of  small  nodules  may  be  reduced  and  this  can  result  in  a  corresponding  reduction  in  detection 
performance  [11].  As  combined  slices  become  thicker  by  adding  more  of  the  originally  acquired  thin  slices  the 


1 


contrast  of  smaller  nodules,  which  often  are  visible  on  only  one  or  two  thin  slices,  may  be  reduced  by  averaging  their 
voxel  values  with  voxel  values  in  slices  not  containing  the  nodules. 

Over  the  last  decade,  maximum  intensity  projection  (MIP)  has  gained  extensive  attention  for  its  effectiveness  in 
projecting  thicker  volumes  of  CT  data,  especially  angiographic  image  data  [  12] [  1 3] .  This  method  was  originally 
introduced  to  overcome  the  contrast  reduction  associated  with  averaging  methods  and  has  been  studied  extensively 
for  this  purpose  [11].  MIP  orthogonally  projects  the  volume  of  the  combined  slices  onto  a  2D  display  by  using  the 
maximum  pixel  intensity  along  each  ray  of  the  projection.  In  cases  where  the  object  being  sought  is  brighter  than 
superimposed  tissues,  then  the  object  will  be  visible  in  the  MIP  image.  While  MIP  can  improve  contrast  in  volumes 
that  are  not  too  thick,  as  the  thickness  of  the  volume  increases  the  probability  that  some  overlying  tissue  will  have  a 
higher  voxel  value  increases,  and  this  can  reduce  contrast  in  MIP  images. 

MIP  is  commonly  used  in  procedures  requiring  the  visualization  of  vascular  structures  because  of  its  ability  to 
clearly  delineate  bright  objects  overlaying  a  darker  background,  without  significant  loss  of  contrast.  In  general,  MIP 
displays  provide  a  mechanism  for  observers  to  specify  the  number  of  individual  slices  to  be  combined  to  form  each 
thick  slab,  and  then  enable  the  observer  to  move  this  virtual  slab  through  the  data  volume. 

While  such  a  rendering  technique  provides  no  information  of  depth  or  of  the  geometrical  relationships  between 
structures  (i.e.,  tissue  superposition),  albeit  the  appearance  of  3D  projections,  it  does  reduce  the  number  of  images 
that  must  be  viewed  in  order  to  cover  the  entire  3D  volume.  However,  the  images  it  produces  are,  in  principle,  in 
conflict  with  plausible  psychophysical  transparency/brightness  and  geometric  vision  models.  Note  that,  in  order  to 
make  it  possible  for  observers  to  appreciate  the  3D  structure  of  displayed  volumes,  these  displays  sometimes  allow 
the  volume  to  be  continuously  rotated,  though  it  is  known  that  this  kind  of  image  motion  actually  reduces  the 
effective  image  resolution  as  perceived  by  viewers. 

It  is  likely  that  higher  resolution  CT  imaging,  which  will  produce  more  and  thinner  slices,  will  be  adopted  for  lung 
cancer  screening  in  the  future,  and  this  will  exacerbate  problems  associated  with  large  data  volumes  as  well  as  those 
due  to  limited  signal-to-noise  ratios  in  single  slices.  If  helical  CT  is  to  eventually  be  used  for  screening  for  lung 
nodules,  as  many  have  suggested  [14],  [15],  then  the  efficiency  and  accuracy  of  the  interpretation  process  must  be 
increased. 

We  have  proposed  the  use  of  stereoscopic  3D  displays  for  reading  lung  CT  images  as  a  possible  means  to  alleviate 
the  aforementioned  problems,  in  hope  that  this  will  increase  both  efficiency  and  detection  performance  beyond  what 
can  be  achieved  with  other  display  methods  [16],  [17].  Such  displays,  which  have  the  potential  to  provide  more 
natural  representations  and  volume-based  views  of  structures  for  viewers  having  normal  binocular  vision,  have  been 
studied  in  the  past  for  the  display  of  medical  images  under  various  circumstances  but,  due  to  technical  limitations 
(e.g.,  computational  power,  display  technology),  these  methods  have  not  been  widely  adopted.  However,  the  current 
trend  toward  acquisition  of  large  3D  datasets  combined  with  improvements  in  the  relevant  technologies  for 
stereographic  display  make  it  prudent  to  reconsider  the  potential  role  of  these  displays  for  radiology,  at  the  present 
time. 

Lung  CT  images  are  well  suited  for  stereo  viewing  because,  by  making  air  transparent,  the  sparsely  distributed  lung 
tissues  of  interest  (e.g.,  vessels,  airways  and  nodules)  can  be  easily  visualized.  Stereoscopic  displays  have  the 
potential  to  increase  efficiency  and  signal-to-noise  ratios  by  enabling  the  display  of  thicker  tissue  volumes  but  they 
do  not  introduce  the  tissue  superposition  ambiguities  that  are  often  associated  with  a  monoscopic  presentation  of 
thicker  tissue  volumes. 

For  CT  data  to  be  displayed  stereoscopically,  a  3D  dataset  must  be  projected  onto  two  views,  corresponding  to 
observers'  left  and  right  eyes,  using  a  geometric  perspective  transformation  which  projects  rays  through  the  3D 
volume  onto  individual  pixels.  This  process  produces  images  that  simulate  what  each  eye  would  see  if  the  viewer 
were  looking  directly  at  the  3D  dataset.  The  algorithm  that  calculates  a  pixel  value  from  intensity  values  along  a  ray, 
a  process  often  referred  to  as  compositing,  determines  contrast  in  the  final  projected  image.  Choosing  a  compositing 
method  that  is  optimal  for  a  particular  task  is  the  key  decision  that  must  be  addressed  in  optimizing  stereographic 
displays. 
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Compositing  methods  are  generally  subdivided  into  those  that  are  related  to  volume  rendering  (i.e.,  methods  capable 
of  showing  internal  structure)  and  those  based  on  surface  rendering.  Volume  rendering  is  the  most  appropriate 
method  for  nodule  detection  in  lung  CT  images,  due  to  its  ability  to  represent  internal  voxels  and  to  the  fact  that  no 
segmentation  of  surfaces,  which  could  introduce  artifacts,  is  required.  Within  the  general  paradigm  of  volume 
rendering,  there  are  three  compositing  methods  that  appear  to  have  some  applicability  to  rendering  chest  CT  data. 
These  include  distance-weighted  averaging,  conventional  MIP  and  distance-weighted  MIP. 

For  most  applications,  averaging  voxel  values  along  rays  within  the  projected  volume  (i.e.,  averaging  several  CT 
slices  to  create  a  thicker  image),  with  or  without  some  form  of  distance-weighting,  is  the  most  commonly  used 
method  for  compositing  stereo  volumetric  images  from  individual  slices,  owing  to  its  preservation  of  the  internal 
properties  of  objects.  The  drawback  of  the  averaging  method,  as  discussed  above,  is  the  decrease  in  contrast  of  small 
objects  with  increasing  volume. 

Because  of  its  ability  to  largely  solve  the  contrast  problem  in  many  situations,  MIP  has  been  widely  adopted  for  the 
monoscopic  display  of  thicker  slabs.  However,  it  is  not  a  priori  clear  that  MIP  is  suitable  for  stereo  compositing 
because  of  its  inability  to  preserve  the  local  geometric  features  and  texture  of  objects  and,  to  a  large  extent,  it  is  this 
local  information  that  is  required  for  stereopsis.  To  achieve  stereopsis  from  two  views,  a  viewer  needs  to  detect 
corresponding  features  in  each  view  and  to  determine  the  relative  geometric  disparity  between  those  features. 
Features  that  appear  in  one  view  (i.e.,  are  a  maximum  along  projections  onto  the  view)  may  not  be  represented  in  a 
different  view  if  they  are  not  of  maximum  intensity  along  a  projection  for  that  view,  and  such  features  may  provide 
misleading  geometric  disparity  information.  This  causes  small  bright  objects  to  have  a  specular  appearance. 
However,  MIP  does  preserve  the  presence,  but  not  the  exact  geometry,  of  sparsely  distributed  clusters  of  bright 
voxels  when  they  are  viewed  against  a  darker  background,  which  is  essentially  the  situation  that  usually  occurs  when 
a  nodule  is  displayed  in  a  thick  slab  from  axial  CT  slices  of  the  lung.  For  these  reasons,  images  produced  by  MIP 
are,  in  principle,  in  conflict  with  plausible  psychophysical  transparency/brightness  and  geometric  vision  models. 

Distance  adjusted  MIP  is  an  attempt  to  combine  conventional  MIP  with  a  psychophysically  plausible  depth  cue, 
while  retaining  the  contrast  advantages  of  MIP.  In  this  method,  the  brightness  of  voxels  is  reduced  as  a  function  of 
their  depth  within  each  projected  volume.  Brightness  as  a  function  of  depth  is  a  cue  that  most  individuals 
subconsciously  use  all  the  time.  In  viewing  objects  in  any  transparent  medium  that  has  a  constant  optical  density  per 
unit  of  thickness,  the  brightness  of  an  object  will  vary  according  to  an  exponential  function  of  depth.  MIP  displays 
using  this  sort  of  depth  cue  vary  the  brightness  throughout  slabs  as  they  are  moved  through  the  tissue  volume  to 
provide  a  plausible  simulation  of  the  optical  properties  corresponding  to  uniform  optical  density.  This  process  has 
the  added  benefit  for  MIP  that  it  can  change  the  relative  brightnesses  of  voxels  based  on  the  position  of  slabs 
containing  the  voxels,  which  can  improve  the  detectability  of  objects  that  are  similar  in  brightness  to  their 
backgrounds. 

These  various  alternatives  for  compositing  have  not  been  previously  compared  in  the  context  of  nodule  detection  in 
chest  CT,  and  the  degree  to  which  the  concerns  mentioned  above  could  affect  tasks  such  as  nodule  detection  in  lung 
CT  images  is  unknown.  This  paper  explores  these  compositing  methods  with  the  goal  of  identifying  methods 
suitable  for  the  stereographic  display  of  lung  CT  for  the  purpose  of  nodule  detection. 


MATERIALS  AND  METHODS 
1.  Lung  CT  images  and  nodule  verification 

Ten  sets  of  lung  CT  images,  each  containing  a  consensus-proven  solitary  lung  nodule,  were  used  in  this  study.  For 
each  image  set,  stereo  image  pairs  were  projected  using  four  compositing  methods  including  distance-weighted 
averaging,  MIP  and  two  versions  of  distance-weighted  MIP.  The  images  were  displayed  with  window/level  setting 
of  1800/-600  for  MIP  images  and  1500/-600  for  averaging.  Contrast  and  nodule  characteristics  were  compared 
between  these  compositing  methods. 
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1.1.  Lung  CT  images 

Helical  CT  images  were  obtained  from  a  low-dose  CT  lung  cancer  screening  project  conducted  in  the  Medical 
Center,  University  of  Pittsburgh.  The  CT  cases  were  performed  on  a  LightSpeed  Plus  multislice  CT  scanner  (GE 
medical  Systems,  Milwaukee,  WI)  using  X-ray  tube  current  of  40-mA,  voltage  of  140-kVp  and  0.5-mm  pitch.  The 
images  were  acquired  in  the  axial  plane  and  reconstructed  to  a  thickness  of  2.5  mm/slice  with  GE  standard 
convolution  software  for  lung  tissue.  The  pixel  size  in  each  slice  is  0.75-mm  x  0.75-mm. 

1.2.  Nodule  verification 

All  the  nodules  used  in  this  paper  were  identified,  verified,  marked  and  characterized  by  three  experienced 
radiologists.  The  verification  process  was  repeated  at  least  one  time  to  ensure  agreement  on  the  nodule  identification 
and  characterization.  Table  1  lists  the  nodules  and  their  properties. 

2.  Stereo  compositing  methods 

The  voxel  shape  in  the  CT  images  used  in  this  study  was  nonisotropic  in  that  images  had  been  reconstructed  to  a 
larger  thickness  in  z  direction  (i.e.,  slice  thickness  of  2.5-mm)  than  in  x  and  y  directions  (pixel  dimension:  0.75  x 
0.75-mm).  To  approximate  isotropic  voxels,  three  slices  were  created  by  trilinear  interpolation  between  each  pair  of 
adjacent  CT  slices.  Consecutive  slices,  including  CT  and  interpolated  slices,  within  a  given  volume  were  used  for 
generating  a  stereo  pair. 

In  the  conventional  geometric  perspective  transformation  that  was  used  to  compose  left-eye  and  right-eye  image 
pairs,  we  adopted  an  interpupilary  distance  of  6.5-cm,  a  viewing  distance  (the  distance  between  eyes  and  the  screen) 
of  45-cm  and  a  display  area  of  25-cm  x  25-cm.  The  perspective  transformation  was  symmetrical,  based  on  the 
assumption  that  a  viewer  is  centered  in  front  of  the  screen. 

Each  set  of  slices  was  composited  to  a  stereo  pair  for  each  of  the  projection  methods.  Since  the  effect  of 
superimposed  tissue  is  different  for  each  of  the  compositing  methods,  the  characteristics  of  a  projection  method 
depend  on  the  depth  of  a  nodule  within  a  3D  volume.  Thus,  in  order  for  us  to  compare  the  depth-dependence  of 
compositing  methods,  for  each  nodule  we  constructed  three  volumes  with  the  nodule  at  different  depths  in  each 
volume.  Specifically,  the  three  volumes  were  comprised  of:  1)  precisely  those  slices  that  cover  the  nodule;  2)  all 
slices  in  the  first  set  plus  an  additional  3  slices  in  front  of  the  nodule;  and,  3)  all  slices  in  the  first  set  plus  an 
additional  three  slices  behind  the  nodule. 

2.1.  Traditional  Compositing  by  Distance-Weighted  Averaging 

For  this  compositing  method  we  adopted  a  light  emission/transmission/occlusion  model  that  assumed  that  each  voxel 
emits  light  in  proportion  to  its  brightness  when  the  CT  images  are  displayed  at  a  normal  window  and  level  for  nodule 
detection,  and  that  uses  distance  information  (distance  weighing  factors)  to  determine  the  amount  of  this  emitted 
light  that  reaches  the  projection  plane. 

Specifically,  it  was  assumed  that  each  slice  has  a  fixed  optical  density  that  reduces  the  brightness  of  slices  lying 
behind  it.  The  total  of  all  distance-weights  was  equal  to  one.  The  ratio  of  the  weights  between  the  last  slice  (the  slice 
with  the  largest  distance  from  screen)  and  the  first  slice  (the  slice  at  screen  level)  controls  level  of  transparency  for  a 
given  volume. 

We  have  studied  a  range  of  these  ratios  for  lung  CT  images,  and  empirically  set  the  ratio  to  0.5  in  order  to  achieve  a 
balance  between  the  use  of  brightness  weighting  as  a  depth  cue  and  the  visibility  of  the  back  slice.  The  final  value 
for  a  voxel  is  the  sum  of  distance-weighted  pixel  values  in  a  perspective  transformation  ray.  The  detailed 
calculations  were  described  in  references  [16]  and  [17]. 

2.2.  Stereographic  MIP  Compositing 

This  compositing  method  uses  a  perspective  projection  in  which  the  maximum  value  along  each  ray  is  used  as  the 
projected  value.  Because  the  projected  voxels  may  be  different  between  the  stereo  views,  it  is  not  generally  possible 
for  an  observer's  vision  system  to  unambiguously  match  corresponding  points  between  the  two  views.  This  may 
cause  small  objects,  such  as  lung  nodules  to  have  a  speckled  appearance.  Thus,  this  method  is  not  a  priori  suitable 
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for  stereo  projection.  In  practice,  we  have  found  that  the  ambiguity  in  matching  corresponding  points  between  views 
primarily  affects  fine  detail  and  does  not  interfere  with  the  detection  of  objects  that  are  comprised  of  large  clusters  of 
bright  voxels,  though  the  exact  shapes  of  these  objects  cannot  be  unambiguously  determined. 

2.3.  Distance-Weighted  Stereographic  MIP 

In  an  attempt  to  incorporate  a  geometric  cue  common  in  traditional  stereo  projection  methods,  but  also  preserve  the 
contrast  advantage  of  MIP,  we  employed  a  perspective  projection  in  which  the  maximum  along  each  ray  is  weighted 
based  on  distance.  Since  MIP  only  takes  one  pixel  value  along  a  ray,  the  sum  of  total  weight  is  irrelevant  in  this  case 
and  therefore,  only  the  maximum  (nearest)  and  the  minimum  (farthest)  weights  were  empirically  determined.  Based 
on  transparency/occlusion  model,  the  first  slice  was  weighted  as  1,  the  last  slice  was  weighted  as  0.5  and  weights  for 
slices  between  the  first  and  the  last  were  calculated  using  a  geometric  sequence,  assuming  a  fixed  optical  density  for 
each  slice. 


There  are  two  paradigms  for  applying  distance-weighting  to  MIP.  Voxel  values  can  either  be  adjusted  by  distance 
weights  prior  to  acquiring  the  maximum  value  along  a  ray  (Distance-MIP),  or  alternatively,  the  maximum  value  can 
be  chosen  first  and  then  adjusted  by  applying  the  distance-weighting  factor  (MIP-Distance).  In  this  study,  we  have 
tried  both  strategies  and  a  comparison  was  made  based  on  image  appearances  and  contrast  measurements. 


3.  Contrast  analysis 

The  boundaries  of  the  identified  nodules  were  manually  marked  on  CT  slices.  The  background  included  the  area  of 
20-mm  from  the  nodule  boundary  in  x,  y  and  z  directions.  Local  contrast  was  measured  using  Michelson  Contrast 
measure  C„„  with  local  maximum  intensity  Lmax  and  minimum  intensity 
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The  one-tailed  Jonckheere  test  was  used  for  testing  for  a  significant  trend  of  increasing  contrast  for  the  group  of 
three  compositing  methods  as  ordered  in  the  statement  of  the  hypothesis.  The  one-tailed  Wilcoxon-Mann-Whitney 
Test  was  used  for  testing  the  significance  of  difference  in  contrast  measures  between  averaging  and  conventional 
MIP,  between  averaging  and  distance-weighted  MIP,  and  between  distance-weighted  MIP  and  conventional  MIP. 


RESULTS 

We  have  compared  averaging,  MIP  and  distance-weighted  MIP  applied  to  various  nodule  types,  sizes  and  locations. 
Figure  1  illustrates  three  sets  of  nodules  to  show  typical  examples  resulting  from  the  three  stereo  compositing 
methods.  Figure  1A  shows  a  solid  nodule  with  smooth  border,  and  Figures  IB  and  1C  show  nonsolid  nodules  with 
spiculated  borders.  As  is  shown  in  Figure  1,  MIP  and  distance-weighted  MIP  produced  higher  local  contrast  than 
compositing  by  averaging,  in  all  subsets.  If  the  area  surrounding  a  nodule  contains  dense  structures,  such  as  bone  in 
this  case,  a  nodule  can  be  camouflaged  by,  or  blended  into,  the  background  when  stereo  pairs  are  rendered  with 
conventional  MIP,  as  shown  in  Figure  IB  and  Figure  1C.  This  camouflage  effect  is  less  noticed  in  the  distance- 
weighted  MIP  images. 

Local  contrast  measures  for  10  nodules  with  different  compositing  methods  are  listed  in  the  Table  2,  where  each  cell 
has  3  sets  of  Michelson  Contrast  numbers.  Two  numbers  in  a  set  are  for  the  left  and  right  images  in  a  stereo  pair,  and 
the  first  set  of  numbers  are  from  the  subset  of  images  containing  a  nodule,  the  second  set  of  the  numbers  are  from  the 
images  containing  the  nodule  plus  extra  front  slices  and  the  third  set  of  the  numbers  are  from  the  images  containing  a 
nodule  plus  extra  back  slices.  In  general,  conventional  MIP  and  distance-weighted  MIP  (both  MIP-Distance  and 
Distance-MIP)  produced  higher  contrast  measures  compared  to  the  averaging  method.  The  contrast  measures  from 
MIP  method  are  fairly  constant  in  all  slab  thickness  used  in  this  study,  while  the  averaging  method  resulted  in 
decreasing  contrast  as  slab  thickness  increased.  There  is  a  statistical  significant  (p<0.01)  of  ordered  contrast  change 
(averaging,  distance-weighted  MIP,  conventional  MIP)  among  the  three  compositing  methods  as  measured  by 
Jonckheere  test.  Also  notice  that  the  MIP-Distance  approach,  in  general,  had  slightly  higher  average  contrast 
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measures  than  the  Distance-MIP  approach.  When  comparing  contrast  between  the  compositing  methods,  the  one-tail 
Wilcoxon-Mann- Whitney  Test  indicated  that  there  was  a  significant  difference  between  averaging  and  distance- 
weighted  MIP  (either  MIP-Distance  approach  or  Distance-MIP  approach,  p<0.05),  between  averaging  and 
conventional  MIP  (p<0.01)  in  all  three  compositing  thicknesses.  There  was  no  statistically  significant  difference  in 
contrast  measure  between  distance-weighted  MIP  and  conventional  MIP  in  all  three  compositing  thicknesses 
(p>0.05),  except  one  pair,  which  was  the  difference  between  Distance-MIP  and  conventional  MIP  applied  to  the 
subgroup  of  nodule  with  three  front  slices  (p=0.0375). 

Finally,  we  have  compared  the  three  compositing  methods  visually  for  nodule  characterization.  As  shown  in  the 
Figures  IB  and  1C,  the  spiculated  nodule  border  is  clearly  visible  in  all  stereo  pairs  composited  with  averaging 
method,  while  this  characteristic  is  not  preserved  in  some  of  the  stereo  pairs  composited  with  conventional  MIP 
method,  especially  those  composited  from  thicker  slabs.  Distance-weighted  MIP  partially  overcomes  the  problem  of 
conventional  MIP,  and  spiculated  borders  are  still  visible  in  the  thicker  slabs.  As  for  the  smooth  border  we  observed 
similar  phenomenon.  The  geometric  relationship  is  well  presented  with  gradient  changes  in  intensities  along  the 
smooth  border  of  the  nodule  (Figure  1A)  in  the  stereo  pair  composited  with  averaging  method.  The  same  nodule  is 
displayed  with  lack  of  geometric  fidelity,  as  shown  by  the  sharp  edge  of  the  smooth  border  in  the  stereo  pair 
composited  with  conventional  MIP. 


DISCUSSION 

Previously,  we  have  shown  that  stereoscopic  display  for  lung  CT  images  can  dramatically  improve  display  efficiency 
and  lung  nodule  visibility,  relative  to  conventional  slice-based  displays.  In  this  present  study,  we  have  further 
investigated  the  effects  of  several  commonly  used  compositing  methods  on  nodule  representation  in  stereo  CT 
images. 

The  results  from  this  study,  as  well  as  from  others,  have  shown  that  unlike  averaging  methods,  which  sacrifice 
contrast  in  order  to  take  account  of  each  voxel  in  a  volume,  the  conventional  MIP  method  is  able  to  retain  contrast  in 
cases  where  the  object  being  viewed  includes  voxels  that  are  brighter  than  the  superimposed  tissue.  As  applied  to  the 
task  of  lung  nodule  detection,  despite  a  lack  of  geometric  fidelity,  the  conventional  MIP  images  generally  produce 
high  local  contrast  that  separates  a  nodule  from  its  background,  and  therefore  enhances  detection  performance. 

The  improvements  in  nodule  visibility  with  the  MIP  method,  however,  do  not  apply  in  certain  cases  in  lung  CT 
images.  For  a  nodule  to  be  detected  with  conventional  MIP,  it  must  contain  some  voxels  that  are  brighter  than  its 
background.  The  case  of  a  nodule  overlying  a  rib  is  a  particular  concern  though  it  occurs  relatively  infrequently  in 
projections  of  axial  slabs,  unless  the  nodule  is  very  close  to  a  rib  or  the  slab  being  viewed  is  relatively  thick.  Most 
voxels  in  such  a  nodule  will  not  be  as  bright  as  voxels  in  the  rib  and  the  nodule  may  be  almost  invisible.  Examples 
from  this  study  (MIP  images  in  Figures  IB  and  1C),  in  which  the  nodules  are  almost  indiscernible  when  they  are 
close  to  bone  tissue,  have  demonstrated  this  effect. 

This  same  potential  problem  occurs  in  monoscopic  MIP  displays  that  are  already  in  wide  use.  The  situation  is 
actually  somewhat  improved  in  the  case  of  stereo  viewing.  Traditional  MIP  employs  an  orthogonal  projection,  the 
rays  of  which  do  not  change  relative  to  the  3D  volume  as  a  slab  is  moved  in  the  axial  direction.  This  means  that 
when  a  voxel  of  a  rib  and  a  voxel  of  a  nodule  fall  on  the  same  ray  of  a  projection,  then  the  two  voxels  will  always  be 
superimposed  when  both  are  contained  within  a  slab.  The  stereographic  projections  we  are  using  involve  two 
perspective  transformations  for  each  axial  position  of  a  slab.  The  angles  of  rays  passing  through  a  nodule,  in  each  of 
these  projections  are  different  so  that  if  two  voxels  are  superimposed  along  one  ray  they  will  not  be  along  the  other. 
Furthermore,  as  a  slab  is  moved  in  the  axial  direction,  the  orientation  of  a  projection  ray  passing  through  a  particular 
voxel  changes  continuously,  so  that  the  sets  of  superimposed  voxels  change  continuously.  Thus,  if  a  slab  is  moved 
slightly,  an  obscured  nodule  is  more  likely  to  become  visible  with  stereographic  projection  as  opposed  to 
monoscopic  MIP. 

One  benefit  of  the  distance  weighted  MIP  projection  is  that  it  can  reduce  the  brightness  of  the  background.  As  the 
position  of  a  slab  is  changed,  the  relative  brightness  weighting  factors  between  voxels  at  different  axial  positions  will 
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change  and,  in  many  instances,  there  will  be  an  axial  position  where  a  nodule  will  appear  brighter  that  a  rib  and  this 
can  increase  the  likelihood  that  an  obscured  nodule  becomes  visible. 

Segmenting  the  ribs  and  spine  can  theoretically  reduce  this  problem,  but  that  process  may  generate  its  own  artifacts. 
Segmentation  would  involve  some  sort  of  surface  detection,  and  if  the  surfaces  were  perfectly  smooth,  that  should 
not  present  any  difficulties.  However,  any  realistic  surface  detection  algorithm  must  differentiate  between  roughness 
of  the  surface  and  nodules  lying  near  the  surface.  If  a  computer  could  do  this  reliably  then  there  would  be  no  need 
for  the  radiologist  to  view  the  images.  Consequently,  we  are  particularly  concerned  about  the  risks  of  segmentation 
in  precisely  those  cases  where  it  could  be  of  potential  value. 

As  described  in  the  methods,  we  have  tried  two  different  ways  for  distance-weighted  M1P  compositing,  the 
Distance-MIP  approach  and  the  MIP-Distance  approach.  The  two  approaches  produced  slightly  different  results  due 
to  the  fact  that  different  maximum  intensities  were  chosen.  For  distance-MIP,  the  maximum  value  Imax  is  dependent 
on  weight  distribution  in  j  elements  Ij, 

/max  =  max(/1w1,/2w2,/3w3,...,/y-wy). 

If  weight  distribution,  wh  w%  w3,  ....  Wj,  drops  fast,  weighted  values  away  from  the  front  are  greatly  reduced,  and  it 
is  more  likely  that  values  nearer  the  front  will  be  chosen.  In  this  case,  the  final  image  may  represent  only  a  few  slices 
in  the  front,  and  volume  information  could  be  lost.  In  this  study,  that  effect  was  not  apparent  because  of  the 
relatively  slow  change  in  the  weighing  factors.  For  the  MIP-Distance  approach,  although  it  may  not  necessarily  have 
the  maximum  intensity  in  the  final  images  as  might  be  the  case  if  a  chosen  voxel  had  a  large  reduction  in  intensity 
because  it  was  located  in  the  back,  it  does  take  into  account  the  entire  volume  and  produces  contrast  generally  as 
good  as  that  with  Distance-MIP.  An  example  is  shown  in  Figure  IB  (3  slices  +  nodule)  in  which  the  geometric 
relationship  was  better  preserved  in  the  images  rendered  with  MIP-Distance  compositing  compared  to  Distance-MIP 
composting.  For  the  purpose  of  detection  and  diagnosis,  the  MIP-Distance  approach  may  be  preferred  to  the 
Distance-MIP  approach. 

Our  results  indicate  that  both  versions  of  distance-weighted  MIP  partially  recover  geometric  information  lost  in 
conventional  MIP,  by  incorporating  a  distance  cue  into  the  compositing.  As  shown  in  the  Figure  1,  distance- 
weighted  MIP  reveals  the  nodule  within  the  bone  area  (Figures  IB  and  1C),  where  the  same  nodule  is  not  easy  to 
detect  in  the  conventional  MIP  projection.  However,  the  distance-weighted  MIP  had  image  contrast  nearly 
equivalent  to  that  produced  by  the  conventional  MIP  method.  Overall,  distance-weighted  MIP  retained  most  of  the 
contrast  of  MIP,  while  improving  geometric  fidelity. 

In  addition  to  detection,  the  nodule  characteristics  are  essential  and  critical  for  clinically  differentiating  benign  from 
malignant.  Although  the  conventional  MIP  is  superior  for  detection,  it  was  outperformed  by  the  distance-weighted 
averaging  method  in  terms  of  characterization,  as  can  be  seen  in  our  results.  The  lack  of  fidelity  in  nodule  shape  and 
geometric  representation,  in  the  conventional  MIP  images,  are  attributed  to  the  nature  of  MIP  compositing,  in  which 
the  two  views  may  be  based  on  projections  of  different  voxels.  The  averaging  method,  on  the  other  hand,  has  been 
shown  in  this  study  to  faithfully  retain  the  characteristics  of  the  nodules,  including  structural,  spatial  and  geometric 
information.  This  can  be  observed  in  all  nodules  tested  in  this  study. 

Because  nodules  must  be  detected  before  they  can  be  characterized,  and  we  have  shown  a  tradeoff  between  contrast 
for  detection  and  geometric  fidelity  for  characterization,  we  believe  that  the  use  of  two  separate  stereoscopic  display 
modes  is  both  desirable  and  feasible.  First,  the  image  data  should  be  displayed  using  some  form  of  distance-weighted 
MIP  to  maximize  detection  performance.  Once  a  nodule  has  been  detected,  the  thickness  of  the  displayed  volume 
can  be  adjusted  so  as  to  include  only  those  slices  that  contain  the  nodule,  and  this  volume  can  be  displayed  using  an 
averaging  compositing  method  in  order  that  the  nodule  can  be  more  accurately  characterized. 

Unfortunately,  while  monoscopic  approaches  to  3D  display  can  be  appreciated  by  most  individuals,  there  is  a 
significant  variation  across  the  population  in  the  ability  to  achieve  stereopsis  [18].  It  has  been  reported  that  2%  to  4% 
of  individuals  are  stereopsis  blind  and  another  10%  to  15%  have  a  stereopsis  deficiency  in  the  sense  that  they  have 
difficulty  deriving  3D  information  from  random-dot  stereograms.  Much  of  the  stereopsis  impairment,  reflected  in 
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these  numbers,  has  been  attributed  to  strong  uncorrected  astigmatism  [18].  This  would  seem  to  limit  the  value  of 
stereo  display  to  the  subpopulation  having  normal  binocular  vision.  This  kind  of  limitation  is  not  unprecedented  in 
radiology  -  colorblind  radiologists  would  likely  have  difficulty  taking  advantage  of  the  benefits  of  using  color  on  a 
PET  display,  for  example,  and  previously,  medical  student  applicants  for  a  stereo  X-ray  fluoroscopy  training 
program  have  been  prescreened  using  random-dot  stereograms.  Nevertheless,  for  the  majority  of  radiologists, 
stereographic  methods  may  provide  a  more  natural  way  for  them  to  perceive  the  spatial  relationships  in  3D 
volumetric  datasets,  and  those  radiologists  who  are  unable  to  achieve  stereopsis  would  be  able  to  continue  to  reading 
from  traditional  monoscopic  displays  or  employ  other  means  of  representing  3D  data. 

There  is  also  a  concern  that  even  individuals  who  can  achieve  stereopsis  on  surface  rendered  images  will  have 
difficulty  seeing  volume  rendered  images  in  stereo,  though  there  is  no  direct  evidence  of  this.  In  any  event,  in  the 
scenario  we  are  studying  with  respect  to  stereographic  projection  of  the  lung,  the  objects  whose  positions  we  are 
trying  to  clarify  are  small  relative  to  the  lung  volume  and  tend  to  be  sparsely  distributed.  In  the  detection  task,  it  is 
not  as  important  that  we  see  the  internal  structures  of  the  objects  in  stereo,  as  it  is  that  we  see  the  relative  positions  of 
the  objects  in  stereo.  We  could  achieve  our  goals  by  surface  rendering  the  interior  of  the  lung,  but  as  mentioned 
above,  that  kind  of  calculation  requires  that  rather  intelligent  emulation  of  the  mental  processing  typical  of 
radiologists,  be  performed  in  software.  This  has  the  risk  of  introducing  other  artifacts,  and  is  not  necessary  to  solve 
the  detection  problem. 

The  advantage  of  applying  stereoscopic  technique  over  other  3D  rendering  techniques  for  medical  3D  data  display  is 
that  stereo  display  employs  a  mechanism  naturally  used  by  human  visual  system  for  detecting  and  characterizing 
objects.  By  presenting  data  in  a  volume-based  stereoscopic  format,  radiologists'  efficiency  and  accuracy  in 
interpreting  CT  images  may  be  significantly  improved.  Although  the  actual  value  of  stereo  display  for  lung  cancer 
screening  is  not  yet  known,  it  was  the  intention  of  this  study  to  begin  to  investigate  methods  for  achieving  optimal 
image  quality  in  anticipation  of  future  observer  performance  studies  aimed  at  measuring  the  efficacy  of  stereo 
displays  for  chest  CT.  It  is  noted  that  this  study  has  involved  a  small  sample  size  of  test  sets,  and  a  much  larger  study 
would  be  required  to  clarify  all  differences  between  the  compositing  methods. 
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Table  1.  Nodule  information. 


Nodule  # 

Size  (mm2) 

Border 

Characteristic 

1 

4x2 

Smooth 

Solid 

2 

8x6 

Spiculated 

Non-solid 

3 

10x7 

Spiculated 

Solid 

4 

7x7 

Spiculated 

Non-solid 

5 

6x4 

Speculated 

Mixture  of  solid  and  non-solid 

6 

6x5 

Spiculated 

Solid 

7 

5x4 

Smooth 

Mixture  of  solid  and  non-solid 

8 

10x9 

Spiculated 

Mixture  of  solid  and  non-solid 

9 

6x5 

Smooth 

Mixture  of  solid  and  non-solid 

10 

6x5 

Smooth 

Mixture  of  solid  and  non-solid 

9 


Table  2.  Contrast  measurements  with  Michelson  Contrast  Measure.  Three  sets  of  measurements  in  each  cell  are  the 
stereo  pairs  (left  and  right)  composited  from  the  nodule  slices,  the  nodule  slices  with  3  front  slices,  and  nodule  slices 
with  3  back  slices,  respectively. 


Nodule  # 

Averaging 

MIP-Distance 

Distance-MIP 

MIP 

1 

0.82/0.84 

0.73/0.71 

0.76/0.74 

0.85/0.85 

0.81/0.79 

0.81/0.81 

0.84/0.85 

0.81/0.77 

0.80/0.80 

0.85/0.86 

0.84/0.83 

0.81/0.83 

2 

0.84/0.84 

0.79/0.83 

0.79/0.78 

0.88/0.86 

0.82/0.87 

0.84/0.85 

0.84/0.84 

0.79/0.83 

0.87/0.85 

0.89/0.88 

0.85/0.90 

0.87/0.85 

3 

0.89/0.84 

0.82/0.80 

0.87/0.80 

0.90/0.86 

0.89/0.89 

0.87/0.85 

0.91/0.85 

0.90/0.88 

0.87/0.80 

0.94/0.89 

0.90/0.89 

0.90/0.88 

4 

0.91/0.92 

0.87/0.87 

0.89/0.88 

0.92/0.93 

0.89/0.88 

0.91/0.92 

0.92/0.93 

0.90/0.88 

0.90/0.91 

0.94/0.95 

0.90/0.91 

0.93/0.94 

5 

0.82/0.84 

0.77/0.80 

0.79/0.83 

0.85/0.87 

0.82/0.84 

0.83/0.87 

0.86/0.89 

0.81/0.84 

0.82/0.87 

0.88/0.91 

0.85/0.86 

0.85/0.88 

6 

0.91/0.93 

0.90/0.89 

0.89/0.88 

0.97/0.96 

0.93/0.92 

0.92/0.92 

0.96/0.96 

0.93/0.92 

0.93/0.93 

0.99/0.98 

0.98/0.96 

0.98/0.98 

7 

0.76/0.91 

0.75/0.79 

0.73/0.77 

0.82/0.91 

0.79/0.80 

0.76/0.79 

0.78/0.90 

0.79/0.79 

0.76/0.77 

0.87/0.96 

0.81/0.85 

0.77/0.83 

8 

0.93/0.93 

0.90/0.90 

0.91/0.92 

0.96/0.96 

0.94/0.95 

0.92/0.91 

0.96/0.95 

0.94/0.94 

0.94/0.94 

0.99/0.99 

0.98/0.99 

0.98/0.97 

9 

0.78/0.80 

0.72/0.73 

0.75/0.75 

0.81/0.86 

0.80/0.80 

0.82/0.84 

0.80/0.82 

0.79/0.80 

0.86/0.86 

0.87/0.88 

0.82/0.82 

0.86/0.86 

10 

0.92/0.93 

0.83/0.82 

0.85/0.84 

0.93/0.94 

0.86/0.91 

0.91/0.91 

0.93/0.94 

0.86/0.84 

0.91/0.90 

0.96/0.96 

0.90/0.91 

0.92/0.91 
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Figure  1.  Stereo  image  pairs  of  three  nodule  sets  (A.,  B.,  and  C.).  Images  of  A,  B,  and  C  are  from  nodule  #1,  nodule 
#2  and  nodule  #4,  respectively  as  denoted  in  the  table  1  and  table  2.  In  the  row  labels  of  each  set,  "Nodule"  means 
the  images  were  composited  from  nodule  slices,  "3  slices  +  Nodule"  means  the  images  were  composited  from  nodule 
slices  plus  3  front  slices,  and  "Nodule  +  2  slices"  means  the  images  were  composited  from  nodule  slices  plus  3  back 
slices. 
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Appendix  B. 

Real-Time  Stereographic  Display  of  Volumetric  Datasets  in  Radiology 

Xiao  Hui  Wang,  Glenn  S.  Maitz,  J.  Ken  Leader,  Walter  F.  Good 

A  workstation  for  testing  the  efficacy  of  stereographic  displays  for  applications  in  radiology  has  been  developed,  and 
is  currently  being  tested  on  lung  CT  exams  acquired  for  lung  cancer  screening.  The  system  exploits  pre-staged 
rendering  to  achieve  real-time  dynamic  display  of  slabs,  where  slab  thickness,  axial  position,  compositing  method, 
brightness  and  contrast  are  interactively  controlled  by  viewers.  Stereo  viewing  is  by  means  of  shutter-glasses 
synchronized  to  a  144  Hz  monitor.  The  system  enables  viewers  to  toggle  between  alternative  renderings  such  as  one 
using  distance-weighted  ray-casting  by  maximum-intensity -projection,  which  is  optimal  for  detection  of  small 
features  in  many  cases,  and  ray-casting  by  distance-weighted  averaging,  for  characterizing  features  once  detected. 

A  reporting  mechanism  is  provided  which  allows  viewers  to  use  a  stereo  cursor  to  measure  and  mark  the  3D 
locations  of  specific  features  of  interest,  after  which,  a  pop-up  dialog  box  appears,  into  which  findings  can  be 
entered. 

In  the  past,  stereo  displays  have  been  underutilized  in  radiology  as  a  result  of  a  perception,  which  developed  prior  to 
the  widespread  adoption  of  inherently  3D  imaging  modalities,  that  they  provide  only  a  minimal  benefit  to 
experienced  radiologists.  Because  there  is  no  consensus  with  regard  to  the  efficacy  of  stereoscopic  displays  in 
radiology,  the  system's  impact  on  performance  is  being  tested  on  chest  CT  exams  for  lung  cancer  screening.  These 
exams  are  uniquely  suited  for  testing  the  potential  of  stereographic  display  methods  in  radiology.  On  each  slice, 
radiologists  must  determine  whether  small  round  spots  correspond  to  a  vessel  passing  through  the  slice  or  to  a  cross- 
section  of  a  nodule  -  a  task  that  is  very  dependent  on  the  3D  information  in  adjacent  slices.  In  addition  to  this 
objective  study  of  performance,  radiologists'  subjective  assessments  have  been  solicited  for  other  kinds  of  3D  exams 
(e.g.,  breast  MRI).  Radiologists'  subjective  responses  to  the  system  have  been  positive,  but  objective  estimates  of 
changes  in  performance  and  efficiency  must  await  the  conclusion  of  our  study. 

We  are  in  the  process  of  implementing  the  rendering  calculations  on  the  computer's  graphics-processing  unit  so  that 
they  can  be  performed  in  real  time.  This  will  greatly  increase  the  system's  flexibility  in  having  users  interactively 
specify  views. 
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Appendix  C. 


Instructions  for  Observers 

Introduction  -  This  study  is  designed  to  measure  the  relative  performances  of  different  display  methods  used  for 
detecting  nodules  in  lung  CT  exams.  Its  goal  is  to  evaluate  any  association  between  display  method,  performance 
and  efficiency  in  reading  chest  CT  exams  acquired  for  lung  cancer  screening. 

Task  -  You  will  be  shown  a  series  of  lung  CT  exams  and  asked  to  identify,  and  mark  the  locations  of,  features  that 
correspond  to  possible  nodules  having  sizes  >  3-mm.  Your  confidence  as  to  the  likelihood  that  a  detected  feature  is  a 
nodule,  and  your  assessment  of  its  characteristics,  are  to  be  entered  into  a  dialog  box  that  pops  up  when  a  feature  is 
marked.  All  cases  will  be  viewed  in  one  of  three  display  modes  on  a  CRT  monitor. 

Definition  of  a  possible  nodule  -  For  the  purposes  of  this  study  a  nodule  will  be  defined  as  a  round  opacity,  at  least 
moderately  well  marginated  and  no  greater  than  3 -cm  in  maximum  diameter.  This  would  not  include  obvious  linear 
scars  or  nonspecific  opacities  without  definite  boundaries.  However,  marking  such  abnormalities,  and  then  entering  a 
low  rating  for  likelihood  of  being  a  nodule,  may  be  an  appropriate  response  in  certain  cases.  These  cases  have  a 
relatively  high  concentration  of  nodules.  There  may  be  more  than  one  nodule  in  a  single  case.  Other  abnormalities 
may  be  present,  but  for  the  purposes  of  this  study,  they  should  be  ignored. 

Display  -  The  workstation  provides  for  three  display  modes,  though  only  a  single  mode  will  be  available  during  any 
given  reading  session.  The  display  modes  include:  1)  traditional  slice-by-slice;  2)  MIP  display  of  thick  slabs;  and,  3) 
stereo  display  of  thick  slabs.  In  both  the  MIP  and  stereo  modes,  when  the  slab  thickness  is  set  to  1 ,  the  displays  are 
equivalent  to  the  slice-by-slice  mode. 

All  user  interaction  will  be  through  the  use  of  the  computer's  mouse  and  a  specialized  keypad.  You  will  use  keypad 
keys  to  navigate  through  slices  and  to  adjust  the  display's  window  and  level.  In  the  MIP  and  Stereo  display  modes, 
slab  thickness  and  axial  position  can  be  controlled  through  the  keypad.  The  locations  of  suspected  nodules  are  to  be 
marked  by  clicking  on  the  displayed  image.  The  mouse  is  also  to  be  used  to  answer  questions  on  the  dialog  box  that 
pops  up  when  a  possible  nodule  is  marked.  There  is  no  time  restriction  for  evaluating  a  case.  Detailed  instructions 
for  each  mode  are  provided  below. 


General  Instructions  for  All  Display  Modes 

Marking  the  Locations  of  Possible  Nodules  -  The  locations  of  possible  nodules  are  to  be  marked  by  moving  the 
crosshair  cursor  over  the  nodule  and  left-clicking.  A  box  will  be  displayed  on  the  image,  centered  on  the  marked 
location.  At  the  same  time,  a  scoring  form  will  pop  up  on  the  right  hand  side  of  the  screen.  Once  the  scoring  form 
has  been  completed,  the  box  is  replaced  by  a  small  triangular  cue,  and  it  is  possible  to  proceed  with  the  case. 

Measuring  Sizes  -  The  cross-hair  cursor  measures  1-cm  x  1-cm,  relative  to  the  dimensions  of  the  image  data,  and 
can  be  used  to  estimate  the  sizes  of  features,  before  they  are  marked.  Once  a  location  is  marked,  a  2-cm  x  2-cm  box 
(with  tic  marks  5-mm  long)  is  placed  around  the  location  while  the  scoring  form  is  visible,  and  can  be  used  for 
estimating  sizes. 

Answering  Questions  on  the  Scoring  Form  -  Answer  the  questions  on  the  scoring  form  based  on  your  assessment 
of  the  marked  location.  The  1st  question  on  the  scoring  form  relates  to  your  subjective  estimate  of  the  likelihood  that 
a  nodule  is  present  at  the  marked  location;  with  0  indicating  that  you  believe  there  is  definitely  not  a  nodule,  and  100 
indicating  that  you  are  certain  a  nodule  is  present.  The  second  question,  which  relates  to  the  likelihood  that  the 
nodule  is  malignant,  is  to  be  answered  similarly.  In  both  cases,  it  is  important  that,  as  far  as  possible,  you  use  the 
entire  scale  when  you  express  your  actual  level  of  confidence  for  each  detection.  In  particular,  do  not  to  be 
excessively  dogmatic  in  the  sense  of  reducing  most  ratings  to  yes  or  no  decisions  with  ratings  of  0  or  100%.  Such 
extreme  ratings  should  be  reserved  for  those  cases  in  which  your  confidence  level  is,  in  fact,  extremely  high.  When 
you  have  completed  the  scoring  form  click  on  the  OK  button.  The  scoring  form  will  disappear  and  the  box 
surrounding  the  marked  location  will  be  replaced  by  a  small  triangular  cue. 
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Showing  or  Hiding  Marked  Locations  -  Cues  identifying  the  marked  locations  can  be  displayed  or  hidden  by 
pressing  either  the  Show  Marks  or  Hide  Marks  buttons  on  the  keypad. 

Changing  Answers  or  Deleting  Marked  Locations  -  Answers  for  a  previously  marked  location  in  the  current  case 
can  be  changed,  or  the  marked  location  deleted,  by  clicking  on  the  cue  identifying  the  location.  A  scoring  form 
containing  the  previous  answers  will  be  presented.  Answers  can  be  changed  or  the  marked  location  can  be  deleted 
(by  clicking  Delete  button). 

Saving  Answers  and  Proceeding  to  Next  Case  -  After  you  have  marked  all  of  the  locations  you  consider  to  be 
suspicious  for  a  nodule,  your  answers  are  to  be  finalized  and  recorded  by  clicking  on  the  Save  Answers  and  Go  To 
Next  Case  button.  If  there  are  more  cases  available  to  be  read  in  your  current  reading  session,  a  new  case  will  be 
presented. 

Terminating  a  Reading  Session  -  To  terminate  the  reading  session,  click  on  Exit  in  the  drop-down  File  menu 
(upper  left  corner  of  screen).  We  have  not  specified  a  fixed  number  of  cases  for  a  reading  session.  However,  the  total 
number  of  interpretations  is  fixed,  so  the  more  you  complete  in  each  session,  the  sooner  you  will  complete  the  study. 

Window  and  Level  Adjustment  -  The  displayed  window  and  level  can  be  adjusted,  or  set  to  default  values,  by 
using  buttons  on  the  keypad  as  shown  in  the  "Instructions  for  Keypad"  figure. 

Axial  Position  -  In  all  display  modes,  slice  position  within  the  dataset  can  be  set  by  using  buttons  on  the  keypad  as 
shown  in  the  "Instructions  for  Keypad"  figure. 

Specific  Instructions  for  Stereo  Display  Mode 

Use  of  Shutter  Glasses  -  In  the  stereo  display  mode,  it  is  necessary  to  wear  the  shutter  glasses.  The  display  actually 
toggles  between  a  left-eye  view  and  a  right-eye  view  of  the  3-D  data  at  a  rate  of  140  Hz.  When  viewing  thicker 
slices  in  this  mode,  these  two  views  will  appear  to  be  superimposed  on  the  display  if  viewed  without  using  the 
shutter  glasses. 

Adjusting  Slab  Thickness  -  Slab  thickness  can  be  adjusted  by  using  the  blue  buttons  on  the  keypad  as  shown  in  the 
"Instructions  for  Keypad"  figure. 

Viewing  Original  (Unmodified)  Image  Data  -  Viewing  data  at  the  minimum  slab  thickness  (1  slice)  is  equivalent 
to  viewing  the  data  in  the  slice-by-slice  mode. 

Marking  Nodule  Locations  -  When  a  location  is  marked  on  the  image,  and  the  scoring  form  has  been  completed,  a 
small  triangular  cue  is  placed  on  each  visible  slice.  However,  for  the  purpose  of  scoring,  we  assume  the  cue  is 
attached  to  the  center  slice  of  the  slab.  Thus,  to  more  accurately  identify  the  position  of  a  nodule  in  the  axial 
direction  when  the  nodule  is  detected  in  a  thicker  slab,  it  may  be  necessary  to  reduce  the  thickness  of  the  slab  before 
marking  the  nodule's  location. 

Measuring  Sizes  -  The  crosshair  cursor  measures  1-cm  x  1-cm,  relative  to  the  dimensions  of  the  slice  in  which  it  is 
displayed  (middle  slice),  and  can  be  used  to  estimate  the  sizes  of  features  before  they  are  marked.  Once  a  location  is 
marked  and  the  scoring  form  is  visible  a  box,  which  is  placed  around  the  location,  can  be  used  for  estimating  sizes. 
The  box  measures  2-cm  x  2-cm  and  has  5-mm  long  tic  marks,  relative  to  the  dimensions  of  the  middle  slice.  Because 
both  of  these  are  displayed  in  the  middle  slice  of  thick  slabs,  they  will  appear  to  change  size  slightly,  as  slab 
thickness  is  changed,  to  reflect  changes  in  the  dimensions  of  middle  slices  resulting  from  the  stereo  perspective 
transformation. 

Changing  Projection  Method  -  Two  methods  for  stereo  projection  -  MIP  and  averaging  -  are  available  in  the 
stereo  display  mode.  MIP  is  the  default,  and  is  generally  considered  to  be  preferable  for  detection  of  nodules,  but  it 
does  not  preserve  geometric  fidelity.  Averaging  gives  a  more  accurate  representation  of  geometric  detail,  but 
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provides  less  contrast.  We  expect  that  the  highest  performance  will  be  achieved  when  stereo  projection  by  MIP  is 
used  for  detection,  and  the  user  switches  to  projection  by  averaging  only  when  it  is  necessary  to  see  the  geometric 
detail  more  clearly.  Clicking  on  the  buttons  Change  to  MIP  Projection  and  Change  to  Averaging  Projection  are 
used  to  toggle  between  these  two  renderings.  Note  that  the  stereo  projection  method  is  reset  to  MIP  after  each 
scoring  form  is  completed  and  when  a  new  case  is  presented. 

Specific  Instructions  for  MIP  Display  Mode 

Adjusting  Slab  Thickness  -  Slab  thickness  can  be  adjusted  by  using  the  blue  buttons  on  the  keypad  as  shown  in  the 
"Instructions  for  Keypad"  figure. 

Viewing  Original  (Unmodified)  Image  Data  -  Viewing  data  at  the  minimum  slab  thickness  (1  slice)  is  equivalent 
to  viewing  the  data  in  the  slice-by-slice  mode. 

Marking  Nodule  Locations  -  When  a  location  is  marked  on  the  image,  and  the  scoring  form  has  been  completed,  a 
small  triangular  cue  is  placed  on  each  visible  slice.  However,  for  the  purpose  of  scoring,  we  assume  the  cue  is 
attached  to  the  center  slice  of  the  slab.  Thus,  to  more  accurately  identify  the  position  of  a  nodule  in  the  axial 
direction  when  the  nodule  is  detected  in  a  thicker  slab,  it  may  be  necessary  to  reduce  the  thickness  of  the  slab  before 
marking  the  nodule's  location. 

Measuring  Sizes  -  The  cross-hair  cursor  measures  1-cm  x  1-cm,  relative  to  the  dimensions  of  the  image  data,  and 
can  be  used  to  estimate  the  sizes  of  features,  before  they  are  marked.  Once  a  location  is  marked,  a  2-cm  x  2-cm  box 
(with  tic  marks  5-mm  long)  is  placed  around  the  location  while  the  scoring  form  is  visible,  and  can  be  used  for 
estimating  sizes. 
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